-
Notifications
You must be signed in to change notification settings - Fork 127
Allow links to have file://
prefix - but don't write them that way by default
#1489
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1489 +/- ##
==========================================
- Coverage 91.31% 91.22% -0.09%
==========================================
Files 53 53
Lines 7286 7296 +10
Branches 883 887 +4
==========================================
+ Hits 6653 6656 +3
- Misses 453 459 +6
- Partials 180 181 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems fine to me, but looks like windoze is cranky 😢
Ok jeez. Down to the one failing Python 3.10 windows test: ================================== FAILURES ===================================
__________ StacIOTest.test_read_write_collection_with_file_protocol ___________
self = <tests.test_stac_io.StacIOTest testMethod=test_read_write_collection_with_file_protocol>
def test_read_write_collection_with_file_protocol(self) -> None:
collection = pystac.read_file(
"file://" + TestCases.get_path("data-files/collections/multi-extent.json")
)
with tempfile.TemporaryDirectory() as tmp_dir:
dest_href = os.path.join(tmp_dir, "collection.json")
> pystac.write_file(collection, dest_href="file://" + dest_href)
tests\test_stac_io.py:33:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pystac\__init__.py:201: in write_file
obj.save_object(
pystac\stac_object.py:476: in save_object
stac_io.save_json(dest_href, self.to_dict(include_self_link=include_self_link))
pystac\collection.py:581: in to_dict
d = super().to_dict(
pystac\catalog.py:645: in to_dict
"links": [link.to_dict(transform_href=transform_hrefs) for link in links],
pystac\catalog.py:645: in <listcomp>
"links": [link.to_dict(transform_href=transform_hrefs) for link in links],
pystac\link.py:387: in to_dict
"href": self.get_href(transform_href=transform_href),
pystac\link.py:181: in get_href
and self.owner.get_root()
pystac\stac_object.py:325: in get_root
root_link.resolve_stac_object()
pystac\link.py:330: in resolve_stac_object
obj = stac_io.read_stac_object(target_href, root=root)
pystac\stac_io.py:235: in read_stac_object
d = self.read_json(source, *args, **kwargs)
pystac\stac_io.py:206: in read_json
txt = self.read_text(source, *args, **kwargs)
pystac\stac_io.py:283: in read_text
return self.read_text_from_href(href)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pystac.stac_io.DefaultStacIO object at 0x0000023DC7CA3E80>
href = '/D:/a/pystac/pystac/tests/data-files/collections/multi-extent.json'
def read_text_from_href(self, href: str) -> str:
"""Reads file as a UTF-8 string.
If ``href`` has a "scheme" (e.g. if it starts with "https://") then this will
use :func:`urllib.request.urlopen` to open the file and read the contents;
otherwise, :func:`open` will be used to open a local file.
Args:
href : The URI of the file to open.
"""
href_contents: str
if _is_url(href):
try:
logger.debug(f"GET {href} Headers: {self.headers}")
req = Request(href, headers=self.headers)
with urlopen(req) as f:
href_contents = f.read().decode("utf-8")
except HTTPError as e:
raise Exception(f"Could not read uri {href}") from e
else:
href = safe_urlparse(href).path
> with open(href, encoding="utf-8") as f:
E OSError: [Errno 22] Invalid argument: '/D:/a/pystac/pystac/tests/data-files/collections/multi-extent.json'
I guess it is probably the leading slash that is the problem. |
I checked when we can deprecate Python 3.10 and it's not until April (https://numpy.org/neps/nep-0029-deprecation_policy.html) so I'll keep chugging |
file:///
prefix - but don't write them that way by defaultfile://
prefix - but don't write them that way by default
…by default (stac-utils#1489) * Allow reading and writing when href startswith file:/// * Add test of reading and writing to hrefs starting with file:/// * Ensure that file:/// are interpretted as absolute urls * Try to fix windows * Try to fix windows * Try to fix windows * Add some print debugging * Add more print debugging * Add more print debugging * Moved is_absolute to os dependent test * Fix os-dependent test * Strip initial slash and see if it passes * Add more print debugging * Just strip off the leading slash * Only for windows * Fix windows handling --------- Co-authored-by: Pete Gadomski <[email protected]>
Related Issue(s):
related to #1347 milder version of #1472 (that one should wait until v2.0)
Description:
Allow links to start with
file:///
but don't write them that way. Also handles reading/writing to path that is prefixed withfile:///
PR Checklist:
scripts/test
)